Synchronization in a Thread-Pool Model and its Application in Parallel Computing

In this paper, we consider synchronization in a thread-pool model and its application in scientific parallel computing. Consider a server system which processes requests from clients, where requests are either to query or to update a unique shared variable. The server receives a request and passes the request to a job thread. The job thread processes the request and sends a reply back to the client. There are two types of concurrent models to implement such a system: a thread-per-request model and a thread-pool model[2]. In the thread-per-request model, every incoming request causes a new job thread to be spawned to process it. In the thread-pool model, the system maintains a pool (or pools) of pre-spawned job threads. When a new request arrives, the request is passed to a free job thread in the pool. After the request is processed, the thread is returned to the pool to execute another request. The advantages of the thread-pool model include that the system can have a bound on the number of threads and reduce the cost of thread creation. The thread-pool model is often used in embedded systems and web servers. We have developed a synchronization methodology well-suited for the thread-pool model. Synchronization has been well-studied in operating systems and parallel computing. Almost all such synchronization codes rely on Operating System (OS) primitives that block executing threads for synchronization. Such solutions are valid for the thread-per-request model. However, in the thread-pool model where threads are a limited resource, often such solutions are not efficient or even not acceptable. For example, consider the producers/consumers with a bounded buffer problem found in many textbooks[1]. Suppose that there are 10 buffer entries and that the thread pool maintains less than 10 threads. Then, if the first 10 requests are from consumers, the system becomes deadlocked. In our approach, when an execution needs to be blocked, synchronization code releases and returns the executing thread to the thread pool, rather than blocking the thread, so that the thread can execute another job. This is done by returning from one function and calling another function. As a result, switching from one job to another is done extremely fast. We applied our synchronization approach to grid-computation and obtained good results. The results show that our thread-pool model synchronization approach can speed up many applications, such as web servers and embedded systems.